

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  <meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />

  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>块设备与 OpenStack &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/css/custom.css" type="text/css" />

  
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/jquery.js"></script>
        <script src="../../_static/_sphinx_javascript_frameworks_compat.js"></script>
        <script data-url_root="../../" id="documentation_options" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/doctools.js"></script>
        <script src="../../_static/sphinx_highlight.js"></script>
    
    <script type="text/javascript" src="../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../genindex/" />
    <link rel="search" title="Search" href="../../search/" />
    <link rel="next" title="块设备与 CloudStack" href="../rbd-cloudstack/" />
    <link rel="prev" title="Block Devices and Nomad" href="../rbd-nomad/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    <div role="navigation" aria-label="Page navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="../../" class="icon icon-home" aria-label="Home"></a></li>
          <li class="breadcrumb-item"><a href="../">Ceph 块设备</a></li>
          <li class="breadcrumb-item"><a href="../rbd-integrations/">Ceph 块设备与第三方对接</a></li>
      <li class="breadcrumb-item active">块设备与 OpenStack</li>
      <li class="wy-breadcrumbs-aside">
            <a href="../../_sources/rbd/rbd-openstack.rst.txt" rel="nofollow"> View page source</a>
      </li>
  </ul>
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../" class="icon icon-home"> Ceph
          

          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../start/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephadm/">Cephadm</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rados/">Ceph 存储集群</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../">Ceph 块设备</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../rados-rbd-cmds/">基本命令</a></li>
<li class="toctree-l2"><a class="reference internal" href="../rbd-operations/">运维</a></li>
<li class="toctree-l2 current"><a class="reference internal" href="../rbd-integrations/">对接</a><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="../rbd-ko/">内核模块</a></li>
<li class="toctree-l3"><a class="reference internal" href="../qemu-rbd/">QEMU</a></li>
<li class="toctree-l3"><a class="reference internal" href="../libvirt/">libvirt</a></li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-kubernetes/">Kubernetes</a></li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-nomad/">Nomad</a></li>
<li class="toctree-l3 current"><a class="current reference internal" href="#">OpenStack</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#index-1">创建一个存储池</a></li>
<li class="toctree-l4"><a class="reference internal" href="#openstack-ceph">配置 OpenStack 的 Ceph 客户端</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id5">让 OpenStack 使用 Ceph</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id9">重启 OpenStack</a></li>
<li class="toctree-l4"><a class="reference internal" href="#id10">从块设备引导</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-cloudstack/">CloudStack</a></li>
<li class="toctree-l3"><a class="reference internal" href="../iscsi-overview/">LIO iSCSI Gateway</a></li>
<li class="toctree-l3"><a class="reference internal" href="../rbd-windows/">Windows</a></li>
<li class="toctree-l3"><a class="reference internal" href="../nvmeof-overview/">NVMe-oF 网关</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../man/">手册页</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/">APIs</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../radosgw/">Ceph 对象网关</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../monitoring/">监控概览</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/developer_guide/">开发者指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/internals/">Ceph 内幕</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../ceph-volume/">ceph-volume</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../hardware-monitoring/">硬件监控</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <section id="openstack">
<h1>块设备与 OpenStack<a class="headerlink" href="#openstack" title="Permalink to this heading"></a></h1>
<p id="index-0">通过 <code class="docutils literal notranslate"><span class="pre">libvirt</span></code> 你可以把 Ceph 块设备用于 OpenStack ，
它配置了 QEMU 到 <code class="docutils literal notranslate"><span class="pre">librbd</span></code> 的接口。
Ceph 把块设备分块为对象并分布到集群中，
这意味着大个的 Ceph 块设备映像其性能会比独立服务器更好。</p>
<p>要把 Ceph 块设备用于 OpenStack ，必须先安装 QEMU 、
<code class="docutils literal notranslate"><span class="pre">libvirt</span></code> 和 OpenStack 。我们建议用一台独立的物理主机安装
OpenStack ，此主机最少需 8GB 内存和一个 4 核 CPU 。下面的图表描述了 OpenStack/Ceph 技术栈。</p>
<p class="ditaa">
<img src="../../_images/ditaa-33c7ae36551d19d56e43c4fb399307021a3d41d5.png"/>
</p>
<div class="admonition important">
<p class="admonition-title">Important</p>
<p>要让 OpenStack 使用 Ceph 块设备，你必须有相应的 Ceph 集群访问权限。</p>
</div>
<p>OpenStack 里有三个地方和 Ceph 块设备结合：</p>
<ul class="simple">
<li><p><strong>映像：</strong> OpenStack 的 Glance 管理着 VM 的映像。映像相对恒定， OpenStack 把它们当作大块二进制数据、并按需下载。</p></li>
<li><p><strong>卷宗：</strong> 卷宗是块设备， OpenStack 用它们引导虚拟机、
或挂到运行着的虚拟机上。
OpenStack 用 Cinder 服务管理卷宗。</p></li>
<li><p><strong>客座磁盘 (Guest Disk)</strong>: 客座磁盘是客座操作系统的磁盘。
默认情况下，你引导虚拟机时，
它的磁盘在管理程序的文件系统上看起来是个文件
（通常在 <code class="docutils literal notranslate"><span class="pre">/var/lib/nova/instances/&lt;uuid&gt;/</span></code> 下面）。
在 OpenStack Havana 版之前，引导 Ceph 里的虚拟机的唯一方法就是用
Cinder 的 boot-from-volume 功能；而现在没有 Cinder 也可以直接引导 Ceph 里的所有虚拟机了，这样更好，
因为这样你就能通过实时迁移功能执行维护任务了。
另外，如果你的管理程序死掉了，也便于你触发 <code class="docutils literal notranslate"><span class="pre">nova</span> <span class="pre">evacuate</span></code> ，
然后几乎是无缝地恢复所有的虚拟机。
与此同时， <a class="reference internal" href="../rbd-exclusive-locks/#rbd-exclusive-locks"><span class="std std-ref">互斥锁</span></a>
会阻止多个计算节点并行地访问客座磁盘。</p></li>
</ul>
<p>你可以用 OpenStack Glance 把映像存储到 Ceph 块设备中，还可以用
Cinder 来引导映像的写时复制克隆品。</p>
<p>下面将详细指导你安装设置 Glance 、 Cinder 和 Nova ，虽然它们不一定一起用。你可以在本地硬盘上运行 VM 、
却把映像存储于 Ceph 块设备，反之亦可。</p>
<div class="admonition important">
<p class="admonition-title">Important</p>
<p>不建议用 QCOW2 作为虚拟机的磁盘格式。
如果你想引导 Ceph 里的（临时后端或从 volume 引导）虚拟机，
请在 Glance 里使用 <code class="docutils literal notranslate"><span class="pre">raw</span></code> 映像格式。</p>
</div>
<section id="index-1">
<span id="id1"></span><h2>创建一个存储池<a class="headerlink" href="#index-1" title="Permalink to this heading"></a></h2>
<p>默认情况下， Ceph 块设备使用 <code class="docutils literal notranslate"><span class="pre">rbd</span></code> 存储池，你可以用任何可用存储池。但我们建议分别为 Cinder 和 Glance 创建存储池。确保 Ceph
集群在运行，然后创建存储池。</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span> <span class="n">osd</span> <span class="n">pool</span> <span class="n">create</span> <span class="n">volumes</span>
<span class="n">ceph</span> <span class="n">osd</span> <span class="n">pool</span> <span class="n">create</span> <span class="n">images</span>
<span class="n">ceph</span> <span class="n">osd</span> <span class="n">pool</span> <span class="n">create</span> <span class="n">backups</span>
<span class="n">ceph</span> <span class="n">osd</span> <span class="n">pool</span> <span class="n">create</span> <span class="n">vms</span>
</pre></div>
</div>
<p>参考<a class="reference external" href="../../rados/operations/pools#createpool">创建存储池</a>为存储池指定归置组数量，参考<a class="reference external" href="../../rados/operations/placement-groups">归置组</a>确定应该为存储池分配多少归置组。</p>
<p>新建的存储池必须先初始化才能使用，用 <code class="docutils literal notranslate"><span class="pre">rbd</span></code> 工具来初始化此存储池：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">rbd</span> <span class="n">pool</span> <span class="n">init</span> <span class="n">volumes</span>
<span class="n">rbd</span> <span class="n">pool</span> <span class="n">init</span> <span class="n">images</span>
<span class="n">rbd</span> <span class="n">pool</span> <span class="n">init</span> <span class="n">backups</span>
<span class="n">rbd</span> <span class="n">pool</span> <span class="n">init</span> <span class="n">vms</span>
</pre></div>
</div>
</section>
<section id="openstack-ceph">
<h2>配置 OpenStack 的 Ceph 客户端<a class="headerlink" href="#openstack-ceph" title="Permalink to this heading"></a></h2>
<p>运行着 <code class="docutils literal notranslate"><span class="pre">glance-api</span></code> 、 <code class="docutils literal notranslate"><span class="pre">cinder-volume</span></code> 、 <code class="docutils literal notranslate"><span class="pre">nova-compute</span></code>
或 <code class="docutils literal notranslate"><span class="pre">cinder-backup</span></code> 的主机被当作 Ceph 客户端，它们都需要
<code class="docutils literal notranslate"><span class="pre">ceph.conf</span></code> 文件。</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ssh</span> <span class="p">{</span><span class="n">your</span><span class="o">-</span><span class="n">openstack</span><span class="o">-</span><span class="n">server</span><span class="p">}</span> <span class="n">sudo</span> <span class="n">tee</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">ceph</span><span class="o">.</span><span class="n">conf</span> <span class="o">&lt;/</span><span class="n">etc</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">ceph</span><span class="o">.</span><span class="n">conf</span>
</pre></div>
</div>
<section id="ceph">
<h3>安装 Ceph 客户端软件包<a class="headerlink" href="#ceph" title="Permalink to this heading"></a></h3>
<p>在运行 <code class="docutils literal notranslate"><span class="pre">glance-api</span></code> 的节点上你得安装 <code class="docutils literal notranslate"><span class="pre">librbd</span></code> 的 Python 绑定：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">apt</span><span class="o">-</span><span class="n">get</span> <span class="n">install</span> <span class="n">python</span><span class="o">-</span><span class="n">rbd</span>
<span class="n">sudo</span> <span class="n">yum</span> <span class="n">install</span> <span class="n">python</span><span class="o">-</span><span class="n">rbd</span>
</pre></div>
</div>
<p>在 <code class="docutils literal notranslate"><span class="pre">nova-compute</span></code> 、 <code class="docutils literal notranslate"><span class="pre">cinder-backup</span></code> 和 <code class="docutils literal notranslate"><span class="pre">cinder-volume</span></code>
节点上，要安装 Python 绑定和客户端命令行工具：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">apt</span><span class="o">-</span><span class="n">get</span> <span class="n">install</span> <span class="n">ceph</span><span class="o">-</span><span class="n">common</span>
<span class="n">sudo</span> <span class="n">yum</span> <span class="n">install</span> <span class="n">ceph</span><span class="o">-</span><span class="n">common</span>
</pre></div>
</div>
</section>
<section id="id4">
<h3>配置 Ceph 客户端认证<a class="headerlink" href="#id4" title="Permalink to this heading"></a></h3>
<p>如果你启用了 <a class="reference external" href="../../rados/configuration/auth-config-ref/#enabling-disabling-cephx">cephx 认证</a>，需要分别为 Nova/Cinder 和 Glance 创建新用户。命令如下：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span> <span class="n">auth</span> <span class="n">get</span><span class="o">-</span><span class="ow">or</span><span class="o">-</span><span class="n">create</span> <span class="n">client</span><span class="o">.</span><span class="n">glance</span> <span class="n">mon</span> <span class="s1">&#39;profile rbd&#39;</span> <span class="n">osd</span> <span class="s1">&#39;profile rbd pool=images&#39;</span> <span class="n">mgr</span> <span class="s1">&#39;profile rbd pool=images&#39;</span>
<span class="n">ceph</span> <span class="n">auth</span> <span class="n">get</span><span class="o">-</span><span class="ow">or</span><span class="o">-</span><span class="n">create</span> <span class="n">client</span><span class="o">.</span><span class="n">cinder</span> <span class="n">mon</span> <span class="s1">&#39;profile rbd&#39;</span> <span class="n">osd</span> <span class="s1">&#39;profile rbd pool=volumes, profile rbd pool=vms, profile rbd-read-only pool=images&#39;</span> <span class="n">mgr</span> <span class="s1">&#39;profile rbd pool=volumes, profile rbd pool=vms&#39;</span>
<span class="n">ceph</span> <span class="n">auth</span> <span class="n">get</span><span class="o">-</span><span class="ow">or</span><span class="o">-</span><span class="n">create</span> <span class="n">client</span><span class="o">.</span><span class="n">cinder</span><span class="o">-</span><span class="n">backup</span> <span class="n">mon</span> <span class="s1">&#39;profile rbd&#39;</span> <span class="n">osd</span> <span class="s1">&#39;profile rbd pool=backups&#39;</span> <span class="n">mgr</span> <span class="s1">&#39;profile rbd pool=backups&#39;</span>
</pre></div>
</div>
<p>把这些用户 <code class="docutils literal notranslate"><span class="pre">client.cinder</span></code> 、 <code class="docutils literal notranslate"><span class="pre">client.glance</span></code> 和 <code class="docutils literal notranslate"><span class="pre">client.cinder-backup</span></code>
的密钥环复制到各自所在节点，并修正所有权：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span> <span class="n">auth</span> <span class="n">get</span><span class="o">-</span><span class="ow">or</span><span class="o">-</span><span class="n">create</span> <span class="n">client</span><span class="o">.</span><span class="n">glance</span> <span class="o">|</span> <span class="n">ssh</span> <span class="p">{</span><span class="n">your</span><span class="o">-</span><span class="n">glance</span><span class="o">-</span><span class="n">api</span><span class="o">-</span><span class="n">server</span><span class="p">}</span> <span class="n">sudo</span> <span class="n">tee</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">ceph</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">glance</span><span class="o">.</span><span class="n">keyring</span>
<span class="n">ssh</span> <span class="p">{</span><span class="n">your</span><span class="o">-</span><span class="n">glance</span><span class="o">-</span><span class="n">api</span><span class="o">-</span><span class="n">server</span><span class="p">}</span> <span class="n">sudo</span> <span class="n">chown</span> <span class="n">glance</span><span class="p">:</span><span class="n">glance</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">ceph</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">glance</span><span class="o">.</span><span class="n">keyring</span>
<span class="n">ceph</span> <span class="n">auth</span> <span class="n">get</span><span class="o">-</span><span class="ow">or</span><span class="o">-</span><span class="n">create</span> <span class="n">client</span><span class="o">.</span><span class="n">cinder</span> <span class="o">|</span> <span class="n">ssh</span> <span class="p">{</span><span class="n">your</span><span class="o">-</span><span class="n">volume</span><span class="o">-</span><span class="n">server</span><span class="p">}</span> <span class="n">sudo</span> <span class="n">tee</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">ceph</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">cinder</span><span class="o">.</span><span class="n">keyring</span>
<span class="n">ssh</span> <span class="p">{</span><span class="n">your</span><span class="o">-</span><span class="n">cinder</span><span class="o">-</span><span class="n">volume</span><span class="o">-</span><span class="n">server</span><span class="p">}</span> <span class="n">sudo</span> <span class="n">chown</span> <span class="n">cinder</span><span class="p">:</span><span class="n">cinder</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">ceph</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">cinder</span><span class="o">.</span><span class="n">keyring</span>
<span class="n">ceph</span> <span class="n">auth</span> <span class="n">get</span><span class="o">-</span><span class="ow">or</span><span class="o">-</span><span class="n">create</span> <span class="n">client</span><span class="o">.</span><span class="n">cinder</span><span class="o">-</span><span class="n">backup</span> <span class="o">|</span> <span class="n">ssh</span> <span class="p">{</span><span class="n">your</span><span class="o">-</span><span class="n">cinder</span><span class="o">-</span><span class="n">backup</span><span class="o">-</span><span class="n">server</span><span class="p">}</span> <span class="n">sudo</span> <span class="n">tee</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">ceph</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">cinder</span><span class="o">-</span><span class="n">backup</span><span class="o">.</span><span class="n">keyring</span>
<span class="n">ssh</span> <span class="p">{</span><span class="n">your</span><span class="o">-</span><span class="n">cinder</span><span class="o">-</span><span class="n">backup</span><span class="o">-</span><span class="n">server</span><span class="p">}</span> <span class="n">sudo</span> <span class="n">chown</span> <span class="n">cinder</span><span class="p">:</span><span class="n">cinder</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">ceph</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">cinder</span><span class="o">-</span><span class="n">backup</span><span class="o">.</span><span class="n">keyring</span>
</pre></div>
</div>
<p>运行 <code class="docutils literal notranslate"><span class="pre">nova-compute</span></code> 的节点，其进程需要密钥环文件：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span> <span class="n">auth</span> <span class="n">get</span><span class="o">-</span><span class="ow">or</span><span class="o">-</span><span class="n">create</span> <span class="n">client</span><span class="o">.</span><span class="n">cinder</span> <span class="o">|</span> <span class="n">ssh</span> <span class="p">{</span><span class="n">your</span><span class="o">-</span><span class="n">nova</span><span class="o">-</span><span class="n">compute</span><span class="o">-</span><span class="n">server</span><span class="p">}</span> <span class="n">sudo</span> <span class="n">tee</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">ceph</span><span class="o">.</span><span class="n">client</span><span class="o">.</span><span class="n">cinder</span><span class="o">.</span><span class="n">keyring</span>
</pre></div>
</div>
<p>还得把 <code class="docutils literal notranslate"><span class="pre">client.cinder</span></code> 用户的密钥存进 <code class="docutils literal notranslate"><span class="pre">libvirt</span></code> ， libvirt
进程从 Cinder 挂载块设备时要用它访问集群。</p>
<p>在运行 <code class="docutils literal notranslate"><span class="pre">nova-compute</span></code> 的节点上创建一个密钥的临时副本：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span> <span class="n">auth</span> <span class="n">get</span><span class="o">-</span><span class="n">key</span> <span class="n">client</span><span class="o">.</span><span class="n">cinder</span> <span class="o">|</span> <span class="n">ssh</span> <span class="p">{</span><span class="n">your</span><span class="o">-</span><span class="n">compute</span><span class="o">-</span><span class="n">node</span><span class="p">}</span> <span class="n">tee</span> <span class="n">client</span><span class="o">.</span><span class="n">cinder</span><span class="o">.</span><span class="n">key</span>
</pre></div>
</div>
<p>然后，在计算节点上把密钥加进 <code class="docutils literal notranslate"><span class="pre">libvirt</span></code> 、然后删除临时副本：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>uuidgen
457eb676-33da-42ec-9a8c-9293d545c337

cat &gt; secret.xml &lt;&lt;EOF
&lt;secret ephemeral=&#39;no&#39; private=&#39;no&#39;&gt;
    &lt;uuid&gt;457eb676-33da-42ec-9a8c-9293d545c337&lt;/uuid&gt;
    &lt;usage type=&#39;ceph&#39;&gt;
        &lt;name&gt;client.cinder secret&lt;/name&gt;
    &lt;/usage&gt;
&lt;/secret&gt;
EOF
sudo virsh secret-define --file secret.xml
Secret 457eb676-33da-42ec-9a8c-9293d545c337 created
sudo virsh secret-set-value --secret 457eb676-33da-42ec-9a8c-9293d545c337 --base64 $(cat client.cinder.key) &amp;&amp; rm client.cinder.key secret.xml
</pre></div>
</div>
<p>保留密钥的 uuid ，稍后配置 <code class="docutils literal notranslate"><span class="pre">nova-compute</span></code> 要用。</p>
<div class="admonition important">
<p class="admonition-title">Important</p>
<p>在所有节点上都使用 UUID 不必要，
但是从平台一致性的角度看，
最好保持相同的 UUID 。</p>
</div>
</section>
</section>
<section id="id5">
<h2>让 OpenStack 使用 Ceph<a class="headerlink" href="#id5" title="Permalink to this heading"></a></h2>
<section id="glance">
<h3>配置 Glance<a class="headerlink" href="#glance" title="Permalink to this heading"></a></h3>
<p>Glance 可使用多种后端存储映像，要让它默认使用 Ceph 块设备，可以这样配置 Glance 。</p>
<section id="kilo">
<h4>Kilo 及更高版<a class="headerlink" href="#kilo" title="Permalink to this heading"></a></h4>
<p>编辑 <code class="docutils literal notranslate"><span class="pre">/etc/glance/glance-api.conf</span></code> 并把下列内容加到 <code class="docutils literal notranslate"><span class="pre">[glance_store]</span></code> 段下：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">glance_store</span><span class="p">]</span>
<span class="n">stores</span> <span class="o">=</span> <span class="n">rbd</span>
<span class="n">default_store</span> <span class="o">=</span> <span class="n">rbd</span>
<span class="n">rbd_store_pool</span> <span class="o">=</span> <span class="n">images</span>
<span class="n">rbd_store_user</span> <span class="o">=</span> <span class="n">glance</span>
<span class="n">rbd_store_ceph_conf</span> <span class="o">=</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">ceph</span><span class="o">.</span><span class="n">conf</span>
<span class="n">rbd_store_chunk_size</span> <span class="o">=</span> <span class="mi">8</span>
</pre></div>
</div>
<p>关于 Glance 的其它可用选项见 OpenStack Configuration Reference:
<a class="reference external" href="http://docs.openstack.org/">http://docs.openstack.org/</a> 。</p>
</section>
<section id="id6">
<h4>让映像支持写时复制克隆功能<a class="headerlink" href="#id6" title="Permalink to this heading"></a></h4>
<p>注意，这里通过 Glance 的 API 展示了后端位置，所以此选项启用时的入口不能公开访问。</p>
<section id="mitaka-openstack">
<h5>除 Mitaka 以外的其它 OpenStack 版本<a class="headerlink" href="#mitaka-openstack" title="Permalink to this heading"></a></h5>
<p>如果你想让映像支持写时复制克隆功能，还得把下列内容加到 <code class="docutils literal notranslate"><span class="pre">[DEFAULT]</span></code> 段下：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">show_image_direct_url</span> <span class="o">=</span> <span class="kc">True</span>
</pre></div>
</div>
</section>
</section>
<section id="id7">
<h4>禁用缓存管理（任意 OpenStack 版本）：<a class="headerlink" href="#id7" title="Permalink to this heading"></a></h4>
<p>禁用 Glance 缓存管理，以免映像被缓存到 <code class="docutils literal notranslate"><span class="pre">/var/lib/glance/image-cache/</span></code> 下；假设你的配置文件里有 <code class="docutils literal notranslate"><span class="pre">flavor</span> <span class="pre">=</span> <span class="pre">keystone+cachemanagement</span></code></p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">paste_deploy</span><span class="p">]</span>
<span class="n">flavor</span> <span class="o">=</span> <span class="n">keystone</span>
</pre></div>
</div>
</section>
<section id="id8">
<h4>映像属性<a class="headerlink" href="#id8" title="Permalink to this heading"></a></h4>
<p>我们建议你配置如下映像属性：</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">hw_scsi_model=virtio-scsi</span></code>: 添加 virtio-scsi 控制器以获得更好的性能、并支持 discard 操作；</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">hw_disk_bus=scsi</span></code>: 把所有 cinder 块设备都连到这个控制器；</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">hw_qemu_guest_agent=yes</span></code>: 启用 QEMU guest agent （访客代理）</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">os_require_quiesce=yes</span></code>: 通过 QEMU guest agent 向外发送文件系统的 freeze/thaw 调用</p></li>
</ul>
</section>
</section>
<section id="cinder">
<h3>配置 Cinder<a class="headerlink" href="#cinder" title="Permalink to this heading"></a></h3>
<p>OpenStack 需要一个驱动和 Ceph 块设备交互，还得指定块设备所在的存储池名字。
编辑 OpenStack 节点上的 <code class="docutils literal notranslate"><span class="pre">/etc/cinder/cinder.conf</span></code> ，添加：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">DEFAULT</span><span class="p">]</span>
<span class="o">...</span>
<span class="n">enabled_backends</span> <span class="o">=</span> <span class="n">ceph</span>
<span class="n">glance_api_version</span> <span class="o">=</span> <span class="mi">2</span>
<span class="o">...</span>
<span class="p">[</span><span class="n">ceph</span><span class="p">]</span>
<span class="n">volume_driver</span> <span class="o">=</span> <span class="n">cinder</span><span class="o">.</span><span class="n">volume</span><span class="o">.</span><span class="n">drivers</span><span class="o">.</span><span class="n">rbd</span><span class="o">.</span><span class="n">RBDDriver</span>
<span class="n">volume_backend_name</span> <span class="o">=</span> <span class="n">ceph</span>
<span class="n">rbd_pool</span> <span class="o">=</span> <span class="n">volumes</span>
<span class="n">rbd_ceph_conf</span> <span class="o">=</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">ceph</span><span class="o">.</span><span class="n">conf</span>
<span class="n">rbd_flatten_volume_from_snapshot</span> <span class="o">=</span> <span class="n">false</span>
<span class="n">rbd_max_clone_depth</span> <span class="o">=</span> <span class="mi">5</span>
<span class="n">rbd_store_chunk_size</span> <span class="o">=</span> <span class="mi">4</span>
<span class="n">rados_connect_timeout</span> <span class="o">=</span> <span class="o">-</span><span class="mi">1</span>
</pre></div>
</div>
<p>如果你在用 <a class="reference external" href="../../rados/configuration/auth-config-ref/#enabling-disabling-cephx">cephx 认证</a>，还需要配置用户及其密钥
（前述文档中存进了 <code class="docutils literal notranslate"><span class="pre">libvirt</span></code> ）的 uuid ：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">ceph</span><span class="p">]</span>
<span class="o">...</span>
<span class="n">rbd_user</span> <span class="o">=</span> <span class="n">cinder</span>
<span class="n">rbd_secret_uuid</span> <span class="o">=</span> <span class="mi">457</span><span class="n">eb676</span><span class="o">-</span><span class="mi">33</span><span class="n">da</span><span class="o">-</span><span class="mi">42</span><span class="n">ec</span><span class="o">-</span><span class="mi">9</span><span class="n">a8c</span><span class="o">-</span><span class="mi">9293</span><span class="n">d545c337</span>
</pre></div>
</div>
<p>注意：如果你想配置多个 cinder 后端， <code class="docutils literal notranslate"><span class="pre">glance_api_versio</span> <span class="pre">=</span> <span class="pre">2</span></code>
必须放到 <code class="docutils literal notranslate"><span class="pre">[DEFAULT</span></code> 段下。</p>
</section>
<section id="cinder-backup">
<h3>Cinder Backup 的配置<a class="headerlink" href="#cinder-backup" title="Permalink to this heading"></a></h3>
<p>OpenStack Cinder Backup 需要专有守护进程，所以别忘了安装。在你的 Cinder Backup 节点上，编辑 <code class="docutils literal notranslate"><span class="pre">/etc/cinder/cinder.conf</span></code> 并加上：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">backup_driver</span> <span class="o">=</span> <span class="n">cinder</span><span class="o">.</span><span class="n">backup</span><span class="o">.</span><span class="n">drivers</span><span class="o">.</span><span class="n">ceph</span>
<span class="n">backup_ceph_conf</span> <span class="o">=</span> <span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">ceph</span><span class="o">.</span><span class="n">conf</span>
<span class="n">backup_ceph_user</span> <span class="o">=</span> <span class="n">cinder</span><span class="o">-</span><span class="n">backup</span>
<span class="n">backup_ceph_chunk_size</span> <span class="o">=</span> <span class="mi">134217728</span>
<span class="n">backup_ceph_pool</span> <span class="o">=</span> <span class="n">backups</span>
<span class="n">backup_ceph_stripe_unit</span> <span class="o">=</span> <span class="mi">0</span>
<span class="n">backup_ceph_stripe_count</span> <span class="o">=</span> <span class="mi">0</span>
<span class="n">restore_discard_excess_bytes</span> <span class="o">=</span> <span class="n">true</span>
</pre></div>
</div>
</section>
<section id="nova-ceph-rbd">
<h3>让 Nova 对接 Ceph RBD 块设备<a class="headerlink" href="#nova-ceph-rbd" title="Permalink to this heading"></a></h3>
<p>要连接 Cinder 设备（普通块设备或从卷宗引导），必须告诉 Nova （和 libvirt ）
连接时用哪个用户和 UUID ， libvirt 连接 Ceph 集群或与之认证时也会用这个用户：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">libvirt</span><span class="p">]</span>
<span class="o">...</span>
<span class="n">rbd_user</span> <span class="o">=</span> <span class="n">cinder</span>
<span class="n">rbd_secret_uuid</span> <span class="o">=</span> <span class="mi">457</span><span class="n">eb676</span><span class="o">-</span><span class="mi">33</span><span class="n">da</span><span class="o">-</span><span class="mi">42</span><span class="n">ec</span><span class="o">-</span><span class="mi">9</span><span class="n">a8c</span><span class="o">-</span><span class="mi">9293</span><span class="n">d545c337</span>
</pre></div>
</div>
<p>Nova 的 ephemeral 后端也会用这两条配置。</p>
</section>
<section id="nova">
<h3>Nova 的配置<a class="headerlink" href="#nova" title="Permalink to this heading"></a></h3>
<p>要让所有虚拟机直接从 Ceph 引导，必须配置 Nova 的 ephemeral 后端。</p>
<p>我们建议在 Ceph 配置文件里启用 RBD 缓存（从 Giant 起默认启用）；另外，启用管理套接字对于故障排查来说大有好处，给每个使用 Ceph 块设备的虚拟机分配一个套接字有助于调查性能和/或异常行为。</p>
<p>可以这样访问套接字：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ceph</span> <span class="n">daemon</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">run</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">ceph</span><span class="o">-</span><span class="n">client</span><span class="o">.</span><span class="n">cinder</span><span class="mf">.19195.32310016</span><span class="o">.</span><span class="n">asok</span> <span class="n">help</span>
</pre></div>
</div>
<p>要启用 RBD 缓存和管理套接字，确保各个管理程序上的 <code class="docutils literal notranslate"><span class="pre">ceph.conf</span></code> 都包含：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>[client]
    rbd cache = true
    rbd cache writethrough until flush = true
    admin socket = /var/run/ceph/guests/$cluster-$type.$id.$pid.$cctid.asok
    log file = /var/log/qemu/qemu-guest-$pid.log
    rbd concurrent management ops = 20
</pre></div>
</div>
<p>调整这些目录的权限：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">mkdir</span> <span class="o">-</span><span class="n">p</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">run</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">guests</span><span class="o">/</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">log</span><span class="o">/</span><span class="n">qemu</span><span class="o">/</span>
<span class="n">chown</span> <span class="n">qemu</span><span class="p">:</span><span class="n">libvirtd</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">run</span><span class="o">/</span><span class="n">ceph</span><span class="o">/</span><span class="n">guests</span> <span class="o">/</span><span class="n">var</span><span class="o">/</span><span class="n">log</span><span class="o">/</span><span class="n">qemu</span><span class="o">/</span>
</pre></div>
</div>
<p>要注意， <code class="docutils literal notranslate"><span class="pre">qemu</span></code> 用户和 <code class="docutils literal notranslate"><span class="pre">libvirtd</span></code> 组可能因系统不同而不同，前面的实例基于 RedHat 风格的系统。</p>
<div class="admonition tip">
<p class="admonition-title">Tip</p>
<p>如果你的虚拟机已经跑起来了，重启一下就能得到套接字。</p>
</div>
</section>
</section>
<section id="id9">
<h2>重启 OpenStack<a class="headerlink" href="#id9" title="Permalink to this heading"></a></h2>
<p>要激活 Ceph 块设备驱动、并把块设备存储池名载入配置，必须重启相关的
OpenStack 服务。在基于 Debian 的系统上需在对应节点上执行这些命令：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">glance</span><span class="o">-</span><span class="n">control</span> <span class="n">api</span> <span class="n">restart</span>
<span class="n">sudo</span> <span class="n">service</span> <span class="n">nova</span><span class="o">-</span><span class="n">compute</span> <span class="n">restart</span>
<span class="n">sudo</span> <span class="n">service</span> <span class="n">cinder</span><span class="o">-</span><span class="n">volume</span> <span class="n">restart</span>
<span class="n">sudo</span> <span class="n">service</span> <span class="n">cinder</span><span class="o">-</span><span class="n">backup</span> <span class="n">restart</span>
</pre></div>
</div>
<p>在基于 Red Hat 的系统上执行：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">sudo</span> <span class="n">service</span> <span class="n">openstack</span><span class="o">-</span><span class="n">glance</span><span class="o">-</span><span class="n">api</span> <span class="n">restart</span>
<span class="n">sudo</span> <span class="n">service</span> <span class="n">openstack</span><span class="o">-</span><span class="n">nova</span><span class="o">-</span><span class="n">compute</span> <span class="n">restart</span>
<span class="n">sudo</span> <span class="n">service</span> <span class="n">openstack</span><span class="o">-</span><span class="n">cinder</span><span class="o">-</span><span class="n">volume</span> <span class="n">restart</span>
<span class="n">sudo</span> <span class="n">service</span> <span class="n">openstack</span><span class="o">-</span><span class="n">cinder</span><span class="o">-</span><span class="n">backup</span> <span class="n">restart</span>
</pre></div>
</div>
<p>一旦 OpenStack 启动并运行正常，应该就可以创建卷宗并用它引导了。</p>
</section>
<section id="id10">
<h2>从块设备引导<a class="headerlink" href="#id10" title="Permalink to this heading"></a></h2>
<p>你可以用 Cinder 命令行工具从一映像创建卷宗：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">cinder</span> <span class="n">create</span> <span class="o">--</span><span class="n">image</span><span class="o">-</span><span class="nb">id</span> <span class="p">{</span><span class="nb">id</span> <span class="n">of</span> <span class="n">image</span><span class="p">}</span> <span class="o">--</span><span class="n">display</span><span class="o">-</span><span class="n">name</span> <span class="p">{</span><span class="n">name</span> <span class="n">of</span> <span class="n">volume</span><span class="p">}</span> <span class="p">{</span><span class="n">size</span> <span class="n">of</span> <span class="n">volume</span><span class="p">}</span>
</pre></div>
</div>
<p>注意映像必须是 RAW 格式，你可以用 <a class="reference external" href="../qemu-rbd/#running-qemu-with-rbd">qemu-img</a> 转换格式，如：</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">qemu</span><span class="o">-</span><span class="n">img</span> <span class="n">convert</span> <span class="o">-</span><span class="n">f</span> <span class="p">{</span><span class="n">source</span><span class="o">-</span><span class="nb">format</span><span class="p">}</span> <span class="o">-</span><span class="n">O</span> <span class="p">{</span><span class="n">output</span><span class="o">-</span><span class="nb">format</span><span class="p">}</span> <span class="p">{</span><span class="n">source</span><span class="o">-</span><span class="n">filename</span><span class="p">}</span> <span class="p">{</span><span class="n">output</span><span class="o">-</span><span class="n">filename</span><span class="p">}</span>
<span class="n">qemu</span><span class="o">-</span><span class="n">img</span> <span class="n">convert</span> <span class="o">-</span><span class="n">f</span> <span class="n">qcow2</span> <span class="o">-</span><span class="n">O</span> <span class="n">raw</span> <span class="n">precise</span><span class="o">-</span><span class="n">cloudimg</span><span class="o">.</span><span class="n">img</span> <span class="n">precise</span><span class="o">-</span><span class="n">cloudimg</span><span class="o">.</span><span class="n">raw</span>
</pre></div>
</div>
<p>Glance 和 Cinder 都使用 Ceph 块设备时，此镜像又是个写时复制克隆，就能非常快地创建新卷宗。在 OpenStack 操作板里就能从那个卷宗引导，步骤如下：</p>
<ol class="arabic simple">
<li><p>启动新例程；</p></li>
<li><p>选择与写时复制克隆关联的镜像；</p></li>
<li><p>选中 ‘boot from volume’ ；</p></li>
<li><p>选中你刚创建的卷宗。</p></li>
</ol>
</section>
</section>



<div id="support-the-ceph-foundation" class="admonition note">
  <p class="first admonition-title">Brought to you by the Ceph Foundation</p>
  <p class="last">The Ceph Documentation is a community resource funded and hosted by the non-profit <a href="https://ceph.io/en/foundation/">Ceph Foundation</a>. If you would like to support this and our other efforts, please consider <a href="https://ceph.io/en/foundation/join/">joining now</a>.</p>
</div>


           </div>
           
          </div>
          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
        <a href="../rbd-nomad/" class="btn btn-neutral float-left" title="Block Devices and Nomad" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
        <a href="../rbd-cloudstack/" class="btn btn-neutral float-right" title="块设备与 CloudStack" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>&#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).</p>
  </div>

   

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>